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CAMERA CALIBRATION USING OFF-AXIS ILLUMINATION 
AND VIGNETTING EFFECTS 

BACKGROUND OF THE INVENTION 

One of the most common activities prior to using an imaging device, such as a 
camera, is calibration. Many applications require reasonable estimates of camera 
parameters, especially those that involve structure and motion recovery. 

There is a plethora of prior work on camera calibration. They can be roughly 
classified as weak, semi-strong and strong calibration techniques. 

Strong calibration techniques recover all the camera parameters necessary for 
correct Euclidean (or scaled Euclidean) structure recovery from images. Many of such 
techniques require a specific calibration pattern with known exact dimensions. 
Photogrammetry methods which rely on the use of known calibration points or 
structures are described by D.C. Brown, "Close-range camera calibration", 
Photogrammetric Engineering, 37(8):855-866, August 1971 and R.Y. Tsai, "A versatile 
camera calibration technique for high-accuracy 3D machine vision metrology using off- 
the-shelf TV cameras and lenses", IEEE Journal of Robotics and Automation, RA- 
3(4):323-344, August 1987. Brown, for example, uses plumb lines to recover distortion 
parameters. Tsai uses corners of regularly spaced boxes of known dimensions for full 
camera calibration. 

G. Stein, "Accurate internal camera calibration using rotation, with analysis of 
sources of error", Fifth International Conference on Computer Vision (ICCV95), pages 
230-236, Cambridge, Massachusetts, June 1995 uses point correspondences between 
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multiple views of a camera that is rotated a full circle to extract intrinsic camera 
parameters very accurately. There is also proposed self-calibration techniques such as 
those described by R.I. Hartley "An algorithm for self calibration from several views", 
IEEE Computer Society Conference on Computer Vision and Pattern 
5 Recognition(CVPR f 94), pages 908-912, Seattle, Washington, June 1994, IEEE 

Computer Society, M. Pollefeys et al. 5 "Self calibration and metric reconstruction in 
spite varying and unknown internal camera parameters", International Conference on 
Computer Vision (ICCV98), pages 90-95, Bombay, India, January 1998, IEEE 
Computer Society Press and A. Zisserman et al., "Metric Calibration of a stereo rig", 
10 IEEE Workshop on Representations of Visual Scenes, pages 93-100, Cambridge, 

Massachusetts, June 1995. 
D Weak calibration techniques recover a subset of camera parameters that will 

m enable only projective structure recovery through the fundamental matrix. Faugeras, 

Jfr "What can be seen in three dimensions with an uncalibrated stereo rig", Second 

yj 15 European Conference on Computer Vision (ECCV , 92), pages 563-578, Santa 

n l Margherita Ligure, Italy, May 1992, Springer-Verlag opened the door to this category of 

|L techniques. There are numerous other players in the field, such as Hartley, "In defense 

£ of the 8-point algorithm", Fifth International Conference on Computer Vision 

\f t . (ICCV95), pages 1064-1070, Cambridge, Massachusetts, June 1995, IEEE Computer 

=0 20 Society Press and A. Shashua, "Projective structure from uncalibrated images: Structure 

from motion and recognition", IEEE transactions on Pattern Analysis and Machine 
Intelligence, 16(8):7788-790, August 1994. 

Semi-strong calibration falls between strong and weak calibration; it allows 
structures that are close to Euclidean under certain conditions to be recovered. Affine 
25 calibration described in J.J. Koeriderink et al. "Affine structure from motion", Journal of 

the Optical Society of America A, 8:377-385538, 1991falls into this category. In 
addition, techniques that assume some subset of camera parameters to be known also 
fall into this category. They include the technique discussed in H.C. Longuet-Higgins, 
"A computer algorithm for reconstructing a scene from two projections", Nature, 
30 293:133-135, 1991 and a technique described by Hartley et al., "Estimation of relative 
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camera positions for uncalibrated cameras, Second European Conference on Computer 
Vision (ECCV92) pages 579-587, Santa Margherita, Liguere, Italy, May 1992, 
Springer- Verlag for recovering camera focal lengths corresponding to two views with 
the assumption that all other intrinsic camera parameters are known. 
5 The common thread of all these calibration methods is that they require some 

form of image feature, or registration between multiple images in order to extract 
camera parameters. 

SUMMARY OF THE INVENTION 

We present a camera calibration technique that requires only a flat, textureless 
10 surface, for example, abl ank p iecej)f paper^and uniform illumination. The camera] 

D optical and physical shortcomings are used to extract the camera parameters. 3 

m The image of the textureless surface having uniform illumination is digitized, 

jjf The parameters of the imaging device are computed based on drop off effects due to the 

yj imaging device. The drop off effects may be dependent on an off-axis pixel projection 

=il 15 effect and a vignetting effect. The parameters may also be computed based on a camera 

tilt effect. The parameters of a model are preferably computed by minimizing the 
jz. difference between the digitized image and the model 

! ~; One advantage of the calibration technique is that no special patterns are 

-JB required. The technique provides reasonable estimates of camera parameters and may 

~ 20 be used for applications that may not need accurate camera parameters. For example 

the technique may be used to provide an estimate of the camera parameter for image 

based rendering applications. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The foregoing and other objects, features and advantages of the invention will be 
25 apparent from the following more particular description of preferred embodiments of 

the invention, as illustrated in the accompanying drawings in which like reference 
characters refer to the same parts throughout the different views. The drawings are not 
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necessarily to scale, emphasis instead being placed upon illustrating the principles of the 
invention. 

FIG. 1 is a block diagram of a computer system in which the present invention 
may be used; 

5 FIG. 2 is a flowchart showing the steps for calibrating an imaging device 

according to the principles of the present invention ; 

FIG. 3 is a diagram showing the factors influencing the pixel intensity 
distribution used to calibrate the imaging device; 

FIG. 4A is an illustration of on-axis illumination; 
10 FIG. 4B is an illustration of the off-axis illumination effect; 

FIG. 4C is an illustration of the off-axis effect showing the correspondence 
0 between focal length, off-axis angle and entrance angle for the off-axis illumination 

^] effect shown in FIG. 3; 

rf FIG. 4D is an illustration of the off-axis effect showing the correspondence 

yj 1 5 between the pixel location on the virtual image plane and the distance from the principal 

«1 point; 

^ FIG. 5 A is an illustration an object surface tilted at an angle from the virtual 

£ image plane; 

: r=(A-^ A " FIG. 5 A shows a circle of uniform illumination on the surface of the source 

ffl 20 object 1 

" FIG. 5C shows the foreshortening effect on an image with the object surface 

tilted as shown in FIG. 5A; 

FIG. 5D shows the camera rotation axis of the source object shown in FIG. 5 A; 
FIG. 5E shows the tilt of the source object with respect to the image plane; 
25 FIG. 6A is an illustration of the vignetting effect; 

FIG. 6B is a further illustration of the vignetting effect. 



DETAILED DESCRIPTION OF THE INVENTION 

FIG. 1 is a block diagram of a typical computer system 100 in which the present 
invention is utilized. Included in the computer system 100 are a Central Processing 
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Unit ("CPU") module 108, a memory system 106 and a system bus chip set 1 10 
connected by a processor bus 1 12. The system bus chip set 1 10 is further connected to 
an Input/Output ("I/O") system 104 by a system bus 1 14. An external storage device 
1 16 is connected to the I/O system 104. A calibration model 1 18 is stored in the storage 
5 device 1 16 and also stored in memory 106. A camera 120 is connected to the I/O 

system 10^. The camera 120 digitizes an image of the textureless surface 122. 

FIG. 2 illustrates the steps for calibrating an imaging device, for example a 
camera, according to the principles of the current invention. In step 200 an image of a 
flat, textureless surface with uniform illumination is digitized. The surface may be any 
10 flat, textureless surface such as, a blank sheet of white paper or a white board. 

In step 202 the change of pixel intensity in the digitized image is used to 
□ determine the intrinsic parameters of the imaging device. The intrinsic parameters 

J determined include the focal length, aspect ratio, principal point and skew. The \. 

^ downhill Nelder-Mead algorithm may be implemented to recover the intrinsic I 

yj 15 parameters, or any other similar algorithm may be used. 

pjj FIG. 3 shows the known factors that result in a change of pixel density 

^ distribution 306 in the digitized image. They include off-axis illumination 300, 

p vignetting 302 and camera tilt 304. The off- axis illumination effect 300 is described in 

; ~ conjunction with FIGs. 4A-D. The camera tilt effect 304 is described in conjunction 

y5 20 with FIGs. 5A-C. The vignetting effect 302 is described in conjunction with FIGs. 6A- 

^ B. 

Off-axis illumination 

FIGs. 4A-B is a perspective view of a source object 400, an image plane 402, a 
first lens or entrance pupil 406 and a second lens or exit pupil 408. The source object 
25 400 has a flat, textureless surface. The entrance pupil 406 represents the limiting 

window, or aperture, through which all light rays from the source object 400 must pass. 
The exit pupil 408 represents the limiting aperture through which all light rays to the 
image plane 402 must pass. It is assumed that the entrance pupil 406 and the exit pupil 
408 are circular. Optics between the two pupils are not shown. 
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It is also assumed that the surface properties of the flat textureless source object 
400 are constant throughout and can be approximated as a Lambertian source. A 
Lambertian source is a source for which luminance is independent of direction. It is 
also assumed that illumination is constant throughout the surface of the source object 
400, that is, there are no shadows on the flat, textureless surface of the source object 
400. The source object 400 is perpendicular to the optical axis 404, and the center of 
the source object 400 is on the optical axis 404. 

In FIG. 4 A a source element dA is shown at the center of the source object 400, 
on the optical axis 404. Light rays from source element dA travel on-axis along the 
optical axis 404 to image element dA' in the image plane 402. As light rays travel from 
the source element dA to the image element dA' they are attenuated. The behavior of 
attenuation is optical in nature. The attenuation of the light ray reduces the illumination 
of the image element dA' in the image plane 402. The amount of attenuation of a ray is 
dependent on the distance traveled by the light ray and on the angle between the light 
ray and the optical path 404. 

In FIG. 4B a Source element dA is shown a distance away from the center of the 
source object 300 hplow the optical axis 404. Light rays from the source element dA 
travel off-axis alo/g an off-axis path 308 at a field angle from the optical axis 404. 

The illumination of the image element dA' in the image plane 402 varies across 
the field of view in proportion to the fourth power of the cosine of the angle 0 between 
the light ray and the optical path 404. Thus, the on-axis light ray traveling along the 
optical axis 404 shown in FIG. 4A from source element dA to image element dA 1 with 
an angle equal to 0° has the least amount of attenuation. 

The illumination of the image element dA in FIGs. 4A and 4B is shown to be in 
proportion to the fourth power of the cosine of the field angle 0 as discussed in P. 
Mavrolis and J. McDonald, "Geometric Optics and Optical Design", Oxford University 
Press, 1997, pp. 130-131 as follows: 

The illuminance on-axis (r o ) at the image point dA' is: 
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where L: The radiance of the source at dA, the emitted flux per unit solid angle, 
per unit projected area of the source. 
S: The area of the pupil normal to the optical axis. 
5 M: The magnification. 

R: The distance of dA to the entrance pupil. 
The flux is related to the illuminance by the equation 

dO 

/'=— (2) 
dA' K } 

Combining equations (1) and (2) and substituting for the size of the image point 
10 dA' = M 2 dA results in equation (3) flux for the on-axis image point dA, where dA is an 

infinitely small area in the source. 



^ LdAS 

d ®o=-jjr- ( 3 ) 



The flux for the off-axis image point dA is: 



d ^ _ L(dA cosff)(Scosff) 
(R/cos0) 2 

= dA ^cos 4 0= dA'-^— cos* 6 
R 2 (MR) 2 



(4) 



15 substituting for (1) and (2) in (4) the illuminance of the off-axis image point is: 



7'(^) = / 0 cos 4 6 
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FIG. 4C shows the relationship between the focal length f and an image point u, 
v a distance r from the principal point on the optical axis 404. Using Pythagoras's 
Theorem and the Cosine Rule for right angled triangle, a substitution for Cos 4 0 in 
equation (5) dependent on focal length f is made in equation (6) below. 



r{0) = h 



f 



2 2 2 
+ U +V 



(6) 



FIG. 4D shows the relationship between the co-ordinates u and v of the image 
point on the virtual image plane 402 and the distance r from the principal point. Using 
Pythagoras's Theorem it is seen that r 2 = u 2 + v 2 . Equation (7) below is the resulting 

O equation after substituting for u and v in equation (6) and performing operations to 

ry 10 reduce the equation. 



(l + ('//) 2 ) 2 



Therefore, the attenuation in illumination of the image point dA ? from the object 
element dA is in proportion to the distance from the center of the image on the optical 
axis 404 and the focal length f. The off-axis attenuation factor is represented by p in 
15 Equation (7). 



Camera Tilt Effect 

The off-axis illumination effect described in conjunction with FIGs. 4A-4D 
assumed that the object surface is perpendicular to the optical path 404. The off-axis 
illumination effect described may be extended to include the camera tilt effect. 
20 The camera tilt effect is described in conjunction with FIGs.5A-E. FIG. 5A 

shows a source object 514, a virtual image plane 502 and an imaging device 516 such 
as a camera. FIG. 5B shows a circular area of uniform intensity 504 on the surface of 
the source object 514. FIG. 5C shows the foreshortening effect due to camera tilt on the 
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circular area of uniform intensity 504 at the image plane 514. FIG. 5D shows the source 
object 514 and the camera rotation around a camera rotation axis 500, the broken line in 
the plane of the object 502, at a camera tilt angle x. FIG. 5E shows the perpendicular 
image plane 514 and the object 502 tilted at an angle from the image plane 514 looking 
5 down the axis 500. 

In FIG. 5 A the virtual image plane 5 14 is shown perpendicular to the camera 
516. The source object 502 is shown at an angle to the virtual image plane 514. An 
image point (u, v) is located in the virtual image plane 5 14. FIG. 5D shows the camera 
rotation axis 500 located at an angle % relative to the x-axis of the source object 502. 
10 Two angles are used to indicate the rotation (tilt) of the source object 502. They are the 

angle % relative to the x-axis and angle x representing the amount of rotation around the 
camera rotation axis 500. Using the camera tilt angle and the image plane rotation 
angle the normal to the tilted object, the surface normal, is: 



n T = (sin x sin cos x sin r, cos r) T . (8) 

15 The light ray that passes through the image point (u,v) on the virtual image 514 

has a unit vector, that is, the inter-pixel spacing is equal to one. The vector for the ray 
direction for the point (u,v) is: 



n»=-M >=cosd(— ,— ,l) r . 



|l + ( 7 ) 



r. 2 f f 



(9) 



Combining equation (8) and (9) the foreshortening effect due to the camera tilt is 

20 thus 



no-rtr = cos^cosr 



( . tanr. . ^ 
1H : — (wsin^- vcos^) 



V 



/ 



(10) 
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The off-axis illumination effect described in conjunction with FIGs. 4A-C is combined 
with the foreshortening effect due to camera tilt in Equation (10). To take into 
consideration the foreshortening effect on the off-axis image point dA shown in FIG. 3, 
dACos 0 in Equation 4 is replaced by dA^ n,) . Also the distance to the lens (R/Cos 
5 9) 2 in Equation 4 is replaced by (R/(n$ n,/ Cos x) f . This is computed based on the 

following reasoning: The equation of the tilted image plane, originally R distance away 
from the center of projection, is: 

p-n T =(0,0,R) T rir = Rcosr (11) 

_ The image point (u, v), whose unit vector in space is n$ is the projection of the 

JJ 10 point R^Iq, where R T is the distance of the 3-D point to the point of projection, 

jg Substituting into equation (1 1), results in: 

SJ R cos r 

z ne-nr 

% Incorporating these changes into Equation 5 for the off-axis illumination effect 

yj results in the following equation for the distribution of intensity taking into 

7k 15 consideration the camera tilt effect and the off-axis illumination effect. 



I\0) = l o (n0-nr) 



n#-nt 



cosr 
V J 

tan r 



cos# 



= / 0 cosd 1 + (wsin^f- vcos^) 

v f j 

= />cosV=/>/? 



cos 4 0 (13) 
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Vignetting Rffect 

In an ideal system the entire field of light rays from the source object is 
transmitted through an imaging device to the image. This requires that the lens stop 
cover the entire field of light rays from the source object. In a non-ideal system a 
5 vignetting effect occurs in the image because some of the light rays from the source 

object are obstructed by the lens stops. The vignetting effect on the image is observed 
as a reduction in illumination of image points at the edge of the image caused by the 
geometric effect of the lens stop. The vignetting effect in an imaging device is 
described in conjunction with FIGs. 6 A and 6B. 

10 FIG. 6A shows a source object 400, a sensor plane 402, a first lens 606, a first 

lens stop 600, a second lens 608 and a second lens stop 610. Cones of light rays from 
points P and Q on the source object 400 in object space pass through the first lens 606. 
As shown, all the light rays from point P on the source object 400 pass through the first 
lens 606 and the second lens 608. However, the upper light rays from point Q on the 

15 source object 300 are cut off, or vignetted, by the lens stop 610 adjacent to the second 

lens 608. 

FIG. 6B is used to calculate the vignetting effect due to light rays from point Q 
being obstructed by the lens stop 610 as seen by the second lens 608. The image point 
P' is shown at the center of the first circle 602. The image point Q' is shown at the 

20 center of the second circle (dashed lines) 604. The second circle 604 shows the 

geometrical projection of the lens stop 600 by rays from Q on the source object 400, in 
respect to the second lens stop 610. The area of the shaded areas lost because of the 
vignetting effect are used to calculate the vignetting effect as described in Strong, 
"Concepts of Classical Optics 1 ', pg 248, W.H. Freeman and Co., San Franciso, CA, 

25 1958. The vignetting effect can be expressed as the approximation. 

I vig (0)*{\-ar)l\0) (14) 



where: r is the distance between P 1 and Q' 
a is the vignetting factor to be determined. 
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It is assumed that the vignetting effect is small compared to the off-axis 
illumination effect discussed in conjunction with FIGs. 4A-D. The vignetting effect is 
geometric in nature and the off-axis illumination effect is optical in nature. The pixel 
intensity drop off effect in the image 402 is dependent on the vignetting effect and the 
5 off-axis illumination effect. 

Combined Effect 

Combining all three effects, camera tilt, off-axis illumination and vignetting is 
achieved by combining Equations 13 and 14. This results in the following equation, the 
calibration model 1 18 (FIG. 1): 

p 10 4M = /id-flr¥ (15) 

where a = vignetting effect, 

yy y = camera tilt effect, 

nj P = off-axis illumination effect. 

j; The other camera intrinsic parameters may be computed using the following 

hj 15 equation: 



( \ sVuoriA 
K 0 a jyVorig 



p 

\py) 



(16) 



where: (p x , p y ) is the principal point 
a is the aspect ratio 
s is the skew 

20 (u orig , v orig ) is the original image location relative to the camera image 

center 



The objective function that is to be minimized is thus: 
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^=I(4,,(^)-/o(i-^)^ 2 



(17) 



10 



H 15 



^ 20 



The minimization in Equation 17 minimizes the error between the digitized 
image stored in memory 106 (FIG. 1) and the calibration model 118 (FIG. 1) stored in 
the storage device 116 (FIG. 1) and memory 106 (FIG. 1). The downhill Nelder-Mead 
algorithm may be implemented to recover the intrinsic parameters or any other similar 
algorithm may be used. 

The calibration technique may not be as accurate as other calibration techniques 
but one advantage is that no special patterns are required. The technique provides 
reasonable estimates of camera parameters and may be used for applications that may 
not need accurate camera parameters. For example the technique may be used to 
provide an estimate of the camera parameters for image based rendering applications. 
Image-based rendering techniques use 2-D images for visualizing in 3-D as well as 
editing and manipulating 3-D objects. A set of 2-D images of the object taken from 
different viewpoints is used to provide the 3-D geometric information such as depth 
information to render novel views of the 3-D objects. 

It will be apparent to those of ordinary skill in the art that methods involved in 
the present invention may be embodied in a computer program product that includes a 
computer usable medium. For example, such a computer usable medium may consist of 
a read only memory device, such as a CD ROM disk or conventional ROM devices, or a 
random access memory, such as a hard drive device or a computer diskette, having a 
computer readable program code stored thereon. 

While this invention has been particularly shown and described with references 
to preferred embodiments thereof, it will be understood by those skilled in the art that 
various changes in form and details may be made therein without departing from the 
spirit and scope of the invention as defined by the appended claims. 



